Stochastic Bounds for Censored Markov Chains

Censored Markov chains (CMC) allow to represent the conditional behavior of a system within a subset of observed states. They provide a theoretical framework to study the truncation of a discrete-time Markov chain when the generation of the state-space is too hard or when the number of states is too large. But the stochastic matrix of a CMC may be difficult to obtain. Dayar et al. (2006) have proposed an algorithm, called DPY, that computes a stochastic bounding matrix for a CMC with a smaller complexity with only a partial knowledge of the chain. We prove that this algorithm is optimal for the information they take into account. We also show how some additional knowledge on the chain can improve stochastic bounds for CMC.